A rate-distortion optimization model for SVC inter-layer encoding and bitstream extraction

The emerging Scalable Video Coding (SVC) Extension of H.264/AVC standard enables partial extraction and decoding of video bitstreams, and thus allows viewing devices to adapt their video reception and playback according to devices capability and network performance. This desirable feature, however, comes with a caveat: the parts of a SVC bitstream needed for good quality playback at different devices may differ significantly depending on the visual characteristics of video contents, the quantization and dependency settings at the SVC encoders as well as the display formats and the error concealment mechanisms adopted by the viewing devices. In this paper, we present the results of our investigation on the intricate relations among these factors. We discovered a set of constraints on the setting of quantization parameters Qp and the choices of reference layers for inter-layer dependencies that ensures good rate-distortion (R-D) trade-offs and regular bitstream extraction paths. We called these constraints, the adaptation rules for inter-layer encoding, and their encoding results, the well-adapted SVC layers and bitstreams. We further discovered that bitstream extraction performed by different viewing devices may follow predictable paths if the SVC bitstream is well-adapted and the playback process extracts a complete set of interdependent NAL units at every successive refinement step. This regularity in bitstream extraction enabled us to develop an scalable bitstream extraction algorithm based on local optimization of rate-distortion ratios. A host of experiments was perfromed using the Joint Scalable Video Model 9 (JSVM v.9) of SVC encoder/decoder. Their purpose was to study the effect of Qp settings and reference layer choices on the R-D performance of SVC bitstreams with respect to different video contents, device types, objective vs. subjective distortion measurements and spatiotemporal interpolation algorithms for error concealment. The experimental results based on standard video test sequences are presented in this paper to support our findings.